Sending Three-Dimensional Images over a Network

ABSTRACT

A benefit is obtained for sending digital information over a network. The digital information is representative of three-dimensional images which include photographic images. The receiving computer renders the images from the received digital information, and a viewer at the receiving computer can interactively choose a viewpoint or perspective to view the images on a display of the receiving computer. The benefit to the sender can be monetary compensation and/or increased recognition of the sender, for example.

This is a continuation of application Ser. No. 13/903,505 filed May 28,2013 which was a continuation of application Ser. No. 13/541,173 filedJul. 3, 2012 which was a continuation of application Ser. No. 12/435,840filed May 27, 2009 which was a continuation of application Ser. No.11/445,165 filed Jun. 2, 2006 which was a divisional of application Ser.No. 10/330,181 filed Dec. 30, 2002 which was a continuation of Ser. No.09/494,979 filed Jan. 31, 2000 which was a CIP of Ser. No. 09/258,663filed Feb. 26, 1999.

Applications Ser. Nos. 13/903,505, 13/541,173, 12/435,840, 11/445,165,10/330,181, 09/494,979, and 09/258,663 are hereby incorportated byreference in their entireties.

TECHNICAL FIELD

The present invention relates generally to obtaining a benefit forsending digital information over a communication network.

BACKGROUND INFORMATION

U.S. Pat. No. 4,940,972, describes displaying to a pilot of an aircrafta synthetic image of the ground terrain over which the pilot is flying.U.S. Pat. No. 5,566,073 describes allowing a pilot to preview a routeeither in flight or on the ground. The '073 patent also describesallowing the pilot to take over and try out different flight strategies.The data volume required to display flight routes in accordance with theabove-mentioned patents is very large. The '073 patent describesrepresenting the terrain as polygons in order to save computer storagespace, but even so, the amount of storage on most home computers allowsonly a limited area and/or resolution level to be displayed. The '073patent describes using a CD-ROM to store the required data, but thedelivery of the CD-ROM to home users requires time and changes in theterrain (such as seasonal changes) can require frequent updates of thedata on the CD-ROM.

SUMMARY OF THE INVENTION

It is an object of some aspects of the present invention to providemethods and apparatus for displaying on a remote computer actual imagesof an area as seen from a viewpoint which is chosen interactively.

It is still another object of some aspects of the present invention toprovide methods and apparatus for displaying on a client computerthree-dimensional images stored in a remote server, which are conveyedto the client via a network, preferably the Internet.

It is still another object of some aspects of the present invention toprovide methods and apparatus for streaming data required for renderingthree-dimensional images on a remote computer.

In some embodiments of the present invention, a processor simulatestraveling along a selected route. At substantially any viewpoint alongthe route, the processor displays the view seen from the viewpoint inthree-dimensional real-life images. A user may select at substantiallyeach point along the route the direction of view and may change thedirection dynamically. Preferably, the user controls the speed ofprogress along the route and may stop the progress and/or reversedirection along the course.

Preferably, the user views the three-dimensional images without havingto store a large database of images on the user's processor, and withoutrequiring that the processor have an unusually large memory capacity.Rather, the user connects to a server, which stores all the requireddata, and the user's processor downloads the data dynamically accordingto the data required to display each specific scene. Preferably, theprocessor connects to the server via a communication link, such as theInternet. Preferably, the data is conveyed by a standard modem atsufficient speed for relatively smooth display of the images.

Alternatively or additionally, the required data is downloaded prior tothe displaying of the scene, or the data is retrieved from a CD or othermemory or storage apparatus associated with the processor.

U.S. patent application Ser. No. 08/939,948, which is assigned to theassignee of the present application and is incorporated herein byreference, describes a method of rendering three-dimensional terrainimages. The method allows fast rendering of images by comparison withother methods known in the art. The method of U.S. patent applicationSer. No. 08/939, 948 uses a hierarchical database in which substantiallyeach image is described in a plurality of blocks at different resolutionlevels. The blocks at lower resolution levels include less detail perunit area, while the blocks of higher resolution levels include moredetail per unit area. The additional storage space required in order tostore the additional data required due to the hierarchical structure is“paid back” in improved time performance of the rendering.

There is further provided in accordance with an embodiment of thepresent invention, a method of displaying three dimensional images,including establishing a communication link between a local processorand a server, transferring data blocks describing three-dimensionalimages over the communication link from the server to the localprocessor; and rendering a three-dimensional image at the localprocessor responsive to the data blocks.

There is further provided in accordance with an embodiment of thepresent invention, an apparatus for displaying three dimensional images,including a processor, a communication link between the processor and aserver over which data blocks describing terrain are transferred, and adisplay on which the processor renders three-dimensional terrain imagesresponsive to the data blocks.

In general, the invention relates to obtaining a benefit for sendingdigital information over a network. The digital information isrepresentative of three-dimensional images which include photographicimages. The receiving computer renders the images from the receiveddigital information, and a viewer at the receiving computer caninteractively choose a viewpoint or perspective to view the images on adisplay of the receiving computer. The benefit to the sender can bemonetary compensation or increased recognition of the sender, forexample.

In one aspect, the invention relates to a method of obtaining a benefitfor sending digital information over a communication network. The stepsof this method comprise: (A) providing at least one first database thatincludes digital information representative of three-dimensional imagesincluding photographic images, (B) receiving requests over thecommunication network for at least some of the digital information inthe database, (C) sending the requested digital information over thecommunication network, and (D) receiving a benefit for sending therequested digital information.

In one embodiment, step (C) further comprises providing the digitalinformation which is associated with at least one of a plurality oftravel courses stored in a second database.

In some embodiments, the benefit includes monetary compensation(possibly derived from advertising) and/or increased recognition of anentity that performs steps (A), (B), and /or (C).

In some embodiments, the method further comprises providing at least onesecond database that includes digital information representative of aplurality of travel courses and each travel course is associated with atleast some of the digital information stored in the first databases.

In one embodiment, step (C) further includes sending software to convertthe requested digital information into the images.

In another aspect, the invention features a system for providingthree-dimensional images. The system comprises at least one firstdatabase comprising digital information representing three-dimensionalimages including photographic images, an input module configured toreceive requests over a communication network for at least some of thedigital information in the first database, and an output moduleconfigured to send the requested digital information over thecommunication network. A benefit is received for using the system tosend the requested digital information over the communication networkusing the system.

The system can further comprise at least one second database thatincludes digital information representative of a plurality of travelcourses where each travel course is associated with at least some of thedigital information stored in the first database. The output module canbe configured to send software to convert the requested digitalinformation into the three-dimensional images. The digital informationincluded in the first database can be associated with is at least one ofa plurality of travel courses stored in a second database. The inputmodule can comprise a software program running on a computer server, andthe output module can comprise a software program running on a computerserver.

In still another aspect, the invention relates to a system for sendingdigital information over a communication network. The system comprises afirst database including digital information representative ofthree-dimensional images including photographs. The system alsocomprises a computer server in communication with the first and with thecommunication network and a client computer for allowing an end user tosend requests over the communication network for at least some of thethree-dimensional images and for allowing the end user to receive thedigital information representing the requested three-dimensional images.The system also comprises an e-commerce business for receiving the enduser's requests over the communication network and redirecting the enduser's requests to the computer server so the computer server can sendthe digital information representing the requested three-dimensionalimages to the end user's client computer. The computer server comprisesan input module configured to receive requests over the communicationnetwork for at least some of the digital information in the firstdatabase, and an output module configured to send the requested digitalinformation over the communication network. An operator of the systemreceives a benefit for sending the requested digital information.

In all aspects and embodiments of the invention the benefit that isreceived for sending the requested digital information can includemonetary compensation and/or increased recognition of the operator orother entity involved with sending the requested digital informationover the communication network.

The present invention will be more fully understood from the followingdetailed Is description, including

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a schematic illustration of a system for obtainingcompensation for sending information over a communication network inaccordance with the present invention;

FIG. 2 is a schematic block diagram illustrating the data structure ofimages stored in a database on a pilot training server, in accordancewith a preferred embodiment of the present invention;

FIG. 3 is a flow chart illustrating a method for preparation of a pilottraining database, in accordance with a preferred embodiment of thepresent invention;

FIG. 4 is a schematic block diagram illustrating a flight coursedatabase on a pilot training server, in accordance with a preferredembodiment of the present invention;

FIG. 5 is a schematic block diagram illustrating a processor for viewingthree-dimensional real-life terrain images, in accordance with apreferred embodiment of the present invention;

FIG. 6 is a schematic view of a viewpoint and a scene viewed therefrom,useful in understanding a method of displaying the selected route, inaccordance with a preferred embodiment of the present invention;

FIG. 7 is a flow chart illustrating the actions of a cache manager whilethe processor of FIG. 4 displays a selected route, in accordance with apreferred embodiment of the present invention;

FIG. 8 is a schematic illustration of an image block from the datastructure of FIG. 2, along with its ancestors, useful in understandingthe flow chart of FIG. 8; and

FIGS. 9a and 9b are flow diagrams illustrating obtaining a benefit forsending digital information over a communication network, in accordancewith the present invention.

DESCRIPTION

FIG. 1 is a schematic illustration of a system 300 for obtainingcompensation for sending information over a communication network inaccordance with the present invention. In FIG. 1, the disclosedembodiment of the system 300 includes one or more computer servers 26(one server 26 is shown), a communication network 306 such as theInternet or other computer network at least one image database 40, atleast one course database 60, at least one third-partyelectronic-commerce business or entity (e-commerce business) 304, and atleast one client computer 310. Each client computer 310 typically willinclude at least a processor 20, a visual display 22, and acommunication network interface 24. The e-commerce business 304 includesone or more server computers for communication with the server(s) 26 andthe client computer(s) 310, generally via the network 306. The network306 can include a variety of different and/or similar links includingwireless links.

Digital information representing real-life three-dimensional images isstored in at least one of the image databases 40, and digitalinformation representing a travel course or route is stored in at leastone of the course databases 60. Real-life three-dimensional images arecreated by taking two-dimensional photographic images and, through theuse of the rendering method described in U.S. patent application Ser.No. 08/939,948 or other software and/or rendering schemes/devices,rendering three-dimensional images such that the images include thephotographic images. The computer server 26 is in communication with theimage database(s) 40, the course database(s) 60, and the communicationnetwork 306. The client computer 310 is also in communication with thecommunication network 306 via the communication network interface 24. Inone embodiment, the communication network interface 24 is a standard33,600 kilobyte per second modem. Other modems, faster or slower, mayalso be used. Additionally, other connections may be used for thecommunication interface 24, such as ISDN connections or direct routers.

An end user uses the client computer 310 to request real-lifethree-dimensional images which depict views of a user defined virtualtravel course or which depict views of preset guided tour. The requestis made by contacting an e-commerce business 304 that is incommunication with the communication network 306. The e-commercebusiness 304 redirects the end user's request to the computer server 26,typically in a way that is transparent to the end user. In response tothe end user's request, the computer server 26 retrieves the digitalinformation representing the requested real-life three-dimensionalimages from the image database 40 that depict views of the user definedvirtual travel course or which depict views of a preset guided tour. Thecomputer server 26 transmits the requested digital information to thecomputer client 310 via the communication network 306. The processor 20in the computer client 310 receives the requested digital informationand reconstructs the real-life three-dimensional images and displaysthem on the visual display 22.

Alternatively or additionally, the processor 20 communicates with thecomputer server 26 through a direct communication line. Furtheralternatively or additionally, the processor 20 receives a storage disk28, such as a CD, from those who supply the real-life three-dimensionalimages or from any other distribution source.

The e-commerce business 304 is also in communication with the computerserver 26, either directly or via the communication network 306. Inresponse to the computer server 26 transmitting the user requesteddigital information to the end user, the entity that controls, operates,and/or owns the computer server 26 and/or the data in the databases 40,60 receives some benefit. The benefit may include, but is not limitedto, monetary compensation, and/or increased recognition of the entity bythe public or some group (including increased traffic to the entity'swebsite).

FIG. 2 is a schematic illustration of the data structure of imagesstored in an image database 40 residing on, or in communication with thecomputer server 26, in accordance with a preferred embodiment of thepresent invention. Image database 40 comprises a plurality of imageblocks 42, labeled 42A, 42B, etc., which contain data representingterrain in various areas as would be perceived from different heights.Preferably, substantially all of blocks 42 are of the same data size.Preferably, the size of the blocks is determined according to theexpected rate of transmission of data via modem 24, so that a block 42may be transmitted on the average within a predetermined amount of time,for example, half a second. In a preferred embodiment of the presentinvention, the blocks comprise 256×256 pixels, although the blocks maybe of any other suitable size. Preferably, the blocks are divided intosub-blocks 43 of smaller sizes, such that processors which work withslow modems may download small sub-blocks in case the entire block isnot required. In a preferred embodiment of the present invention, eachblock 42 is divided into sixteen sub-blocks 43 of 64×64 pixels. Eachpixel is preferably represented by a color and an elevation attribute,as is known in the art. Blocks 42 are preferably real-life images ofterrain areas received from airborne or satellite cameras.

Preferably, each sub-block 43 includes an attachment field in whichadditional optional data objects associated with the area covered by thesub-block are described. These objects preferably include, but are notlimited to, labels, annotations, lines and 3D objects. Each object ispreferably accompanied by coordinates which state the position of theobject within sub-block 43. Preferably, the labels are stored in textformat, the lines are stored as vectors, and the 3D objects are storedas polygons, although any suitable storage format may be used.

The objects may be used to represent existing structures which are notviewed sufficiently well when shown as part of the image. Alternativelyor additionally, the structures may be used to overlay virtualstructures on the terrain. For example, it is possible to add plannedbuildings to the terrain and thus see the effect of the buildings on theview. Further alternatively or additionally, the objects may be used tooverlay map symbols and other markings on the terrain. The markings arepreferably overlaid on the view at a constant size regardless of theresolution level of the terrain displayed.

Blocks 42 are preferably stored in the image database 40 in a compressedform using any suitable compression method, such as JPEG. Blocks 42 areclassified in successive resolution levels 44 labeled 44A, 44B, etc.,according to the height from which they view the terrain and, therefore,the level of detail which they include. A plurality of blocks 42A whichbelong to the lowest resolution level 44A, labeled “level 1,” cover thelargest area per block and therefore have the least detail per areaunit. It is noted that the size of the geographical area covered byblocks 42A of “level 1” is dependent on the specific application of theimage database 40 and may be very diverse. For example, in some flightapplications, a single block 42A includes an image of the entire PlanetEarth, while in an atom-simulation application, which shows the terrainof an atom, block 42A shows the entire atom. Blocks 42B of the nextlevel 44B, labeled “level 2,” preferably cover a quarter of the area ofblocks 42A of “level 1”. Thus, for substantially each block 42A, thereexist four blocks 42B which cover the same area. In a similar manner,each successive level 44 comprises blocks 42 which cover a quarter ofthe area of the blocks 42 of the lower resolution level.

Four blocks 55 of a certain level 44C, which cover the same area as ablock 57 of the preceding level 44B, are referred to as descendants ofblock 57. Conversely, block 57 is referred to herein as the parent ofblocks 55. The parent block 59 of block 57 is referred to herein as an“ancestor” of blocks 55, and is said to be of a lower resolution levelthan its descendants. It is noted that in FIG. 2, the lower resolutionlevels appear higher on the page.

Preferably, each block 42 in the image database 40 includes pointers 52to the four descendants of the block. If one or more of the descendantsdoes not exist, a null pointer 56 is preferably used. Preferably, aheader record 54 comprises an index to blocks 42A of “level 1” such thatthe processor 20 can easily find the block 42A which covers a desiredarea.

The number of levels 44 is dependent on the images input to the computerserver 26 and may differ for different areas of terrain. Thus, one block42A may have descendants up to level 12, for example, while anotherblock 42A may have no descendants. It is also noted that the number ofdescendants of each block 42 may have a value different from four, forexample nine, and may differ for different levels 44.

Preferably, each block 42 is referenced using longitudinal andlatitudinal (x,y) coordinates of one of the points in the block, such asthe top right corner pixel, together with the resolution 44 of theblock.

FIG. 3 is a flow chart illustrating a method for preparation of theimage database 40, in accordance with a preferred embodiment of thepresent invention. Preferably, all the steps described in FIG. 3 areperformed automatically by a processor (referred to herein as a Terrabuilder). The Terra Builder may prepare the images online provided theimages are supplied at a sufficient rate. Alternatively or additionally,the Terra Builder operates together with a human operator to achievebetter accuracy in the preparation of the image database 40.

Preferably, the Terra builder receives one or more images of a terrainarea covered by the database 40 (step 200). The images are preferablyreceived in a standard format, such as TIFF or bitmap. The imagespreferably cover adjacent areas or are partially overlapping. Some ofthe images may cover the same area at different levels of resolution.

The processor cuts the image up into blocks 42 and assigns these blockstemporarily to the highest resolution level (step 202). Blocks of lowerlevels of resolution are prepared by eliminating data from the originalblocks (step 204). Preferably, the blocks of lower resolution levels areprepared by decimation, for example, by eliminating odd (or even) rowsand columns from the higher level blocks. Further preferably, the blocksare filtered using a low pass filter, most preferably before thedecimation.

Thereafter, the blocks from different images are aligned relative toeach other, based on the coordinates of the images and themeter-per-pixel resolution values of the images (step 206). Preferably,the coordinates and meter-per-pixel values are received together withthe images. Alternatively, an operator determines and inputs thecoordinate correspondences and meter-per-pixel values by viewing theimages. Further alternatively or additionally, the operator inputs theheights of the terrain for some or all of the pixels.

Blocks 42 are then compressed, for example, using the JPEG compressionmethod (step 208). The operator may add labels, lines, virtualstructures and other objects, before or after the compression (step210). Alternatively or additionally, the processor automatically derivessuch objects from the images. For example, the processor may identifyroads and/or runways in the images and represent them as objects so thatthey appear more clearly in substantially any resolution of display.

It is noted that although the hierarchical structure of the imagedatabase 40 requires extra storage space, relative to a non-hierarchicalrecord of the terrain, the advantages of the hierarchical structurejustify the extra storage space required. The use of the hierarchicalstructure allows faster rendering of the images on the visual display 22and allows fast download of required images at low resolution levels.Optionally, in order to reduce storage requirements, some of blocks 42are stored only in some of resolution levels 44, and when non-existentresolution levels are required, computer server 26 generates therequired block from a descendant block of a higher resolution level.

Preferably, the user of the processor 20 on the client computer 310 isable to add virtual structures and/or other objects to the terraindescribed by database 40, while viewing the terrain. Preferably, a filestored locally on the processor 20 stores descriptions of the virtualstructures added by the user of the processor 20.

FIG. 4 is a schematic block diagram illustrating an example of a coursestored on a course database 60 residing on or in communication with thecomputer server 26, in accordance with a preferred embodiment of thepresent invention. Preferably, the course database 60 includes a catalogfile 62, which lists all the available routes stored in the coursedatabase 60. For each route, the course database 60 preferably includesa list 64 or three-coordinate points 66 which describe the route. Thethree coordinates preferably represent longitudinal, latitudinal, andheight coordinates of the points along the course, as are known in theart of terrain mapping. Preferably, list 64 also includes speed settings68, and/or other flight-relevant data

The routes in the course database 60 preferably include routesterminating in landings at various airports. Using these routes, pilotsmay become familiar with airports to which they are scheduled to fly.Alternatively or additionally, the course database 60 includes routes offlight through difficult access areas, such as deep canyons in whichhelicopters need to fly. Further alternatively or additionally, thecourse database 60 may include routes for training military pilots inapproaching a required target. Further alternatively or additionally,the course database 60 may include land routes through buildings, town,cities, and/or countries

It is noted that other scenarios may be included in the course database60, such as online images from an area of interest. For example, a routemay be used to display a car race, and the points 66 describing theroute may be received online using a GPS from one of the cars in therace. The cars are preferably superimposed on the images, using methodsknown in the art. A user watching the race on a home computer may selectany viewpoint of interest and is not limited to pre-chosen viewpointsselected by others. The user requested real-life three-dimensionalimages received by the client computer 310 arrive in the form of astream of digital data. In order for the images to be viewable on thevisual display 22, they must be reconstructed from the stream of digitaldata by the processor 20 on the client computer 310.

FIG. 5 is a schematic block diagram of the processor 20 on the clientcomputer 310, in accordance with a preferred embodiment of the presentinvention. The blocks are preferably realized as software processesrunning on a general-purpose microcomputer, although dedicated hardwarerealizations are also possible. Preferably, the processor 20 comprises anavigator 70, which keeps track of a viewpoint of a virtual viewer. Theviewpoint preferably follows a predetermined course which a user ofprocessor 20 is supposed to follow. Preferably, the course is receivedfrom course database 60.

Preferably, the navigator 70 sets a default view direction of theviewpoint in the direction of movement along the course. Alternatively,the default view direction is set directly down towards the terrain. Theuser may change the view direction of the viewpoint without moving outof the course. Therefore, there is no compulsory correlation between theflight direction and the view direction. Preferably, the user may changethe speed of motion along the course. Further preferably, the user maymove the viewpoint out of the course in order to view the area aroundthe predetermined course, onto try to find a better course. Preferably,the user controls the direction, speed, altitude and/or any otherparameter of the viewpoint. Specifically, the user may freeze theviewpoint in order to have a better look at the view from a certainpoint or angle.

The processor 20 preferably further comprises a renderer 72, whichcalculates the view from the viewpoint and continuously renders the viewon the visual display 22. The renderer 72 determines the coordinates ofthe pixels it needs in order to render the view and requests thedescriptions of these pixels from a cache manager 74. Preferably, therenderer 72 determines which blocks 42 and/or sub-blocks 43 include therequired pixels. Alternatively, the cache manager 74 determines theidentity of the required blocks 42 and/or sub-blocks 43. Along with eachrequired pixel, block 42, or sub-block 43, renderer 72 preferably statesthe resolution level 44 at which the block is required. The resolutionlevel is preferably determined based on the distance between theviewpoint and the desired pixel or block 42. Further preferably, theresolution level is also dependent on the number of pixels in the imagedisplayed on the visual display 22. Preferably, the resolution levelsare chosen so that an approximate 1:1 ratio is achieved between thenumber of displayed pixels and the number of data pixels. Preferably,the renderer 72 also overlays the objects associated with the renderedsub-blocks 43.

Preferably, cache manager 74 manages a group of blocks 42 and/orsub-blocks 43 in a cache memory 32 of the processor 20, for example inthe main memory of the processor 20, in accordance with a methoddescribed hereinbelow. Alternatively or additionally, the cache memory32 is defined in a local hard disk associated with the processor 20.Thus, even if the processor 20 is shut down, the renderer canimmediately resume operation when the processor 20 is turned on again,at the point it left off, without downloading the data again from thecomputer server 26. Further alternatively or additionally, the processor20 determines areas which are most commonly visited by the user of theprocessor 20, and blocks 42 from these areas are permanently stored inthe local hard disk of the client computer 310. One such preferredapplication involves positioning the processor 20 within a groundvehicle in order to view the surroundings of the vehicle. Since thevehicle is usually located in the same area, the required download timemay be reduced substantially.

It is noted that the term cache memory is used herein generally to referto any relatively small memory which can be accessed rapidly by theprocessor 20 and is used to save data which is most likely to be used bythe processor 20.

The cache manager 74 downloads from the computer server 26 the blocks 42and/or sub-blocks 43 required by renderer 72, if they are not alreadystored in cache memory 32, and meanwhile provides replacement blocksfrom the cache memory. Preferably, the cache manager 74 referencesblocks 42 on the computer server 26 by providing pointers to therequired blocks. The cache manager 74 has the pointers for the lowestresolution level blocks 42A from header record 54. The pointer to adesired block 42 of any other level 44 is preferably taken from theparent block of the desired block, as described above. Therefore, asdescribed hereinbelow, the cache manager 74 preferably always requeststhat the computer server 26 send a block 42 after the cache manager hasreceived its parent block.

As previously stated, after the end user requests a real-lifethree-dimensional image from an e-commerce business 304 via the clientcomputer 310, the request is redirected by the e-commerce business 304to the computer server 26. Preferably, the processor 20 on the clientcomputer 310 establishes one or more communication connections 76 withthe computer server 26 through which blocks 42 are sent to the processor20. Connections 76 are preferably standard TCP connections as are knownin the art, although any other protocol may be used to form theconnection. Preferably, when connections 76 are not in use bringingblocks 42 required by the renderer 72, the cache manager 74 downloadsblocks in the area of the viewpoint to fill cache memory 32. Preferably,the cache manager 74 attempts to fill cache memory 32 with a sufficientnumber of blocks, such that for any view direction of the currentviewpoint, all blocks 42 required by the renderer 72 are stored in cachememory 32. Preferably, the cache manager 74 stores in cache memory 32the same number of blocks 42 in each resolution level 44. Preferably,cache memory 32 stores, for each resolution level 44, between 9 and 36blocks, which are most preferably organized in a square centereddirectly below the location of the viewpoint. In a preferred embodimentof the present invention, cache memory 32 stores sixteen blocks 42organized in a square for each resolution level 44. Blocks 42 arepreferably chosen such that the viewpoint is closest to the center ofthe square, most preferably, as described in the above-mentioned U.S.patent application Ser. No. 08/939,948.

Preferably, blocks 42 are stored in cache memory 32 in the compressedform in which they are received from the computer server 26. Furtherpreferably, the cache manager 74 decompresses the blocks before they areprovided to the renderer 72. Preferably, the cache manager 74 manages,in addition to cache memory 32, an open cache memory 34 in which blocks42 and/or sub-blocks 43 which were decompressed are stored. Preferably,open cache memory 34 is of a size determined according to the amount ofavailable empty storage space on the memory associated with theprocessor 20 and/or the size of the image rendered on the visual display22. Preferably, the user of the processor 20 may adjust the size of opencache memory 34 in order to achieve maximal rendering speed by savingthe time required for decompression. In a preferred embodiment of thepresent invention, open cache memory 34 has a default size sufficient tostore a few hundred decompressed sub-blocks 43.

Preferably, when open cache memory 34 is full, a least recently used(LRU) method is used to determine which sub-block 43 is to be discardedto make room for a new sub-block. A preferred LRU method is described inthe above-mentioned 08/939,948 patent application. Alternatively oradditionally, any other suitable method of memory management may be usedto manage cache memory 32 and/or open cache memory 34.

The renderer 72 uses blocks 42 from cache manager 74 to render therequired view on display 22. Preferably, when the cache manager 74provides a block 42 of a lower resolution level than that requested bythe renderer 72, the renderer 72 uses the provided block to interpolatea higher resolution-level block. Preferably, the higher resolution-levelblock is interpolated using any method known in the art, such asbilinear, fractal, or texture blending.

When the cache manager 74 finishes downloading an additional block of ahigher resolution level from the computer server 26, the block isprovided to the renderer 72, which updates the rendered viewaccordingly. Preferably, when the viewpoint is in motion, the renderer72 updates the view at least ten times per second so that the user has aperception of constant movement, although other rates of update may alsobe used. Preferably, the renderer 72 renders the view each time fromscratch without using previously-rendered views.

Preferably, the renderer 72 is as described in the above mentioned U.S.patent application Ser. No. 08/939,948. Alternatively or additionally,the renderer 72 may operate in accordance with any other method known inthe art. The renderer 72 is preferably implemented entirely in software.Alternatively, the renderer 72 includes a dedicated hardware processor,such as a 3D graphic accelerator, along with a software package runningon general purpose processor 20 which provides block 42 to the dedicatedhardware processor.

FIG. 6 is a schematic view of a viewpoint 80 and a scene 82 viewedtherefrom, used to explain the operation of the renderer 72, inaccordance with a preferred embodiment of the present invention. Scene82 includes areas 84 close to viewpoint 80, for example 1 kilometer fromthe viewpoint. Other areas 86 of scene 82 are far away from viewpoint80, for example, 50-100 kilometers away. Still other areas 85 of scene82 may be at other distances from viewpoint 80. In order to build a reallife image of the view from viewpoint 80, the renderer 72 needs blocks42 from a high resolution level 44 of area 84, such that a group ofbuildings 88 in area 84 are seen in the image as they would be seen from1 kilometer away. However, for area 86, the renderer 72 needs only a lowresolution level block 42 since a group of buildings 90 in area 86 wouldnot be seen from viewpoint 80.

Preferably, the renderer 72 determines the exact blocks needed and callsfor them using their (x,y) coordinates and their resolution level 44.Alternatively or additionally, the renderer 72 specifies, for eachresolution level 44, the coordinates of the boundaries of the necessaryareas, and the cache manager 74 determines the identities of therequired blocks 42. Preferably, when only a small part of a block 42 isrequired, the cache manager 74 orders only the required sub-blocks 43 inorder to save transmission time. On the average, rendering a view imagerequires between about 20 and 200 sub-blocks 43 of various resolutionlevels 44.

FIG. 7 is a flow chart illustrating the actions of the cache manager 74in displaying a selected route by the processor 20, in accordance with apreferred embodiment of the present invention. After downloading thefirst batch of level 1 blocks, as indicated in block 102, the cachemanager 74 moves into a wait state, as indicated by block 100.

Reference is also made to FIG. 8, which is a schematic illustration of ablock 150 (corresponding either to one of blocks 42 or one of sub-blocks43) requested by the renderer 72, and ancestors 152, 154 and 158 of therequested block. When a request for block 150, identified as “x,” andhaving resolution level N, is received from the renderer 72, the cachemanager 74 determines, as indicated in block 104 (FIG. 7), the level jof the highest resolution-level ancestor of block x stored in cachememory 32. If the block 42 itself is stored in cache memory 32 (i.e.,j=N), the block is provided to renderer 72. Otherwise, the highestresolution level ancestor 152 of block x which is stored in cache memory32 is provided to the renderer 72, as indicated in block 106. Asdescribed hereinbelow, the cache manager 74 downloads the rest of theancestors 158 of block x from the computer server 26 in order ofincreasing resolution levels, as indicated by an arrow 156 in FIG. 8. Asthe blocks are received from the computer server 26, they are suppliedto the renderer 72 so that the user sees an image whose resolutionincreases with time.

Thus, if viewpoint 80 is not changed, or is changed relatively slowly,the resolution level of the images displayed by the renderer 72 isslowly increased until the maximal desired resolution is reached.Naturally, if the image database 40 does not include blocks at a desiredlevel of resolution for a certain area, the last block supplied is ofthe highest existing level of resolution for that certain area.

If no ancestor of block x is found in memory 32 (an eventuality notshown in the figure), the level 1 ancestor 154 of the block is orderedfrom the computer server 26, based on the pointer to the level 1ancestor block in header record 54. While waiting for the ordered block,renderer 72 preferably renders a blank block to the display.Alternatively, renderer 72 waits until level 1 ancestor 154 is received.However, it is noted that such cases are very rare, since blocks 42A oflevel 1 cover very large areas, and usually, blocks 42A adjacent to theblock in use are also downloaded and stored in cache memory 32.

If block x itself was not found in memory 32, the cache manager 74 addsto a download queue the block x and all its ancestors 158 of resolutionlevels higher than level j, as indicated by block 108. If all TCPconnections 76 available to processor 20 are in use, the cache manager74 returns to wait state 100 until one of connections 76 is available.If one of connections 76 is available the newly added blocks to thequeue are immediately ordered. Preferably, the cache manager 74 proceedsto send a download order to the computer server 26 for the lowestresolution-level block in the download queue, as indicated by blocks112, 114, 116 and 118. Alternatively or additionally, the download queueis managed by the computer server 26.

Preferably, if more than one block of the lowest resolution level is inthe queue, the last entered block is downloaded (so long as the block isstill within range, as described hereinbelow). The downloaded block isthereafter removed from the queue, either when the download order issent as indicated in block 120, or when the block has been completelyreceived. The cache manager 74 preferably moves back to wait state 100to wait for the completion of the downloading of the block.

Preferably, before a download order for block x is sent to the computerserver 26, the cache manager 74 checks whether the block is stillneeded, as indicated by block 116. Most preferably, the cache manager 74checks whether the block is within a range of the current viewpoint suchthat it would meet the criteria for the cache manager 74 to order it fordownload to cache memory 32. If block x is not within the range of thecurrent viewpoint, the block is not useful for the renderer 72 and istherefore not downloaded. This situation may occur when the viewpointhas changed substantially since block x was put into the download queue.Alternatively or additionally, the cache manager 74 scans the downloadqueue periodically for block orders which are not currently useful andmust be erased from the queue.

When one of TCP connections 76 notifies the cache manager 74 that thetransfer of a block Y has been completed, the cache manager 74 checkswhether the block is currently needed by the renderer 72, as indicatedby block 122. Preferably, the cache manager 74 queries the renderer 72regarding each received block as to whether the renderer 72 currentlyneeds the block. Alternatively or additionally, the cache manager 74maintains a list of blocks for which download orders were sent, andtherefore are needed by the renderer 72. Preferably, the renderer 72notifies the cache manager 74 of blocks it requested and did not receiveand which it does not need any more. Alternatively, each order from therenderer 72 to the cache manager 74 includes all the blocks it needs,and any blocks not in the order are not needed.

If renderer 72 needs the downloaded block (i.e., it was not orderedsolely to fill cache memory 32, as described hereinbelow), it is passedto the renderer 72, as indicated by block 124. Preferably, all thereceived blocks are stored in cache memory 32 for later use, asindicated by block 126. If cache memory 32 is full, a block beyond thepredetermined range from the current viewpoint is discarded, asindicated by block 128. Preferably, the discarded block is the leastrecently used block which is beyond the predetermined range.Alternatively, the discarded block is chosen from the highest resolutionlevel for which there are blocks beyond the predetermined range.

After downloading of a block has been completed, one of connections 76is necessarily not in use. If the download queue is not empty, a blockfrom the queue is downloaded as described hereinabove and indicated inblocks 112, 114, 116 and 118. However, if the queue is empty, the cachemanager 74 fills cache memory 32 with the blocks within the range of thecurrent viewpoint, so that, for any direction of view from the currentviewpoint, there is no need to download further blocks from the computerserver 26.

Preferably, the next block downloaded for filling cache memory 32 isfrom the lowest resolution level for which all the blocks in the rangeof the viewpoint are not already in the cache memory, as indicated inblock 130. Further preferably, the cache manager 74 first downloads theeight blocks surrounding the block which is directly below the currentviewpoint. Alternatively or additionally, the blocks are orderedaccording to the current view direction of the viewpoint.

FIGS. 9a and 9b are flow diagrams illustrating a method for obtaining abenefit for sending information over a communication network, inaccordance with an embodiment of the invention. Referring to FIGS. 9aand 9b , the end user first uses the client computer 310 containing theprocessor 20 to connect to the communication network 306 via thecommunication network interface 24 (step 400). After this connection isestablished, the end user establishes communication with an e-commercebusiness 304 that is also in communication with the communicationnetwork 306 (step 402). The connection with the e-commerce business 304can be established through the use of a communication network browserprogram that is executed by the processor 20 on the client computer 310.Examples of satisfactory communication network browser programs include,but are not limited to, Netscape Navigator and Internet Explorer.

After the connection with an e-commerce business 304 is established, theend user either makes a request for real-life three-dimensional imagesthat depict views of a user defined virtual travel course (step 404) orthe user makes a request for real-life three-dimensional images thatdepict views of a preset guided tour (step 420). If the end user'srequest is for the real-life three-dimensional images that depict viewsof a user defined travel course, the request is redirected by thee-commerce business 304 to the computer server 26 (step 406). When thee-commerce business 304 redirects the user's request, it sends digitalinformation representing the user defined travel course and overlay datato the computer server 26. The overlay data can include, but is notlimited to, digital information representing things such as labels,three-dimensional objects, and billboards. This overlay data can beinserted into the views along the user defined travel course. The userdefined travel course information and the overlay data, once received bythe computer server 26, are converted by the computer server 26 into aformat that is compatible with the navigator 70 running on the clientcomputer 310.

Upon redirection, the processor 20 on the client computer 310established communication with the computer server 26 (step 408).Preferably, if the processor 20 is accessing the computer server 26 forthe first time, the computer server 26 sends the processor 20 a softwarepackage which includes the navigator 70, the renderer 72 and the cachemanager 74 (step 410). The software package can be in the form of anActiveX plug-in sent to the communication network browser running on theprocessor 20. Alternatively or additionally, the user of the processor20 May receive the software package on a compact disk (CD) or on anyother storage media. Preferably, the software package is stored on theclient computer 310 so that the package need be sent only once.

Thereafter, the computer server 26 sends the user defined virtual travelcourse information and overlay data to the navigator 70, preferablyusing one or more of TCP connections 76 (step 412).

Alternatively, if the end user's request is for a preset virtual guidedtour from the e-commerce business 304 (step 420), the e-commercebusiness determines if the proper software (navigator 70, renderer 72and cache manager 74) is running on the processor 20 (step 422). If thissoftware is not present on processor 20 the e-commerce business 304redirects the client computer 310 to the server 26 to download thesoftware (step 424). If the software is already present on processor 20or after the software is downloaded to processor 20, the e-commercebusiness sends the properly formatted preset virtual guided tour courseinformation and overlay data directly to the client computer 310 (step426). In a preset virtual guided tour, the user has the option offollowing the downloaded preset tour course or navigating freely alongany alternate user defined travel course.

After the user defined travel course information or the virtual guidedtour information is received by the processor 20, the navigator 70begins to run on the downloaded course. Concurrently, the cache manager74 is ordered to download one or more, preferably four, level 1 blocks42A surrounding the starting point of the route from the image database40 that is in communication with the computer server 26 (step 414).Thus, the cache manager 74 will have in local cache memory 32 images ofa very large area surrounding the starting point. The cache manager 74can therefore provide to the renderer 72 images which coversubstantially any area, even if only at a low resolution levelinitially. The renderer 72 then reconstructs the images and sends themto the visual display 22 (step 416). Alternatively or additionally, theuser may choose to begin a tour at a specific location, withoutreferring to a specific route.

The user is able to decide whether to display some or all of the objectsassociated with the database. Thus, the user may request to view onlythe images without any of the objects, or to see only objects whichrepresent existing structures. The user is able to request to view theimages with or without labels. Thus, on a first visit to a new area, theuser may view the area with labels which allow easy acquaintance withthe area, while at a second visit, the user may omit the labels in orderto test whether he/she has properly memorized the important labeledlandmarks. Preferably, the user is able to switch between viewing theterrain with and without the objects, so that, for example, the user mayeasily compare the view of a desired area with and without a group ofplanned structures.

In response to the computer server 26 transmitting the digitalinformation representing real-life three-dimensional images to an enduser, the entity that controls, operates, and/or owns the server 26and/or the data in the databases 40, 60 receives some benefit (step418). The benefit can include, but is not limited to, monetarycompensation and/or increased recognition of the entity by the public orsome group ('including increased “traffic” or “hits” to the entity's website). The monetary compensation can be provided to the entity by thee-commerce business 304 or by some other company such as an advertiserassociated with the entity, in that the entity includes the advertiser'sadvertisements within the images sent by the entity to the end user.

Alternatively, an end user can connect to the computer server 26 torequest real-life three-dimensional image depicting views of presetvirtual guided tours. The end user uses the client computer 310 toestablish a connection with the computer server 26. The computer server26 sends the catalog 62 and/or the header record 54 to the processor 20.The end user of the processor 20 chooses a desired route from thecatalog 62, and the navigator 70 downloads the route from the coursedatabase 60 that is in communication with the computer server 26. Thenavigator 70 then begins to run on the downloaded course as describehereinabove.

It will be appreciated that the above-described embodiments relate todisplaying three dimensional real-life images. A variety of embodimentsand uses are within the scope of the invention including, but are notlimited to, displaying of terrain for purposes of real estate trading,travel, education and amusement uses, in which the terrain may be shownat various levels of detail. Furthermore, the terrain is not limited tothe Earth or parts thereof, and may cover other planets (real orvirtual) and/or 3D views of surfaces of real or imaginary objects, suchas views showing the atomic structure of a material, and the like. Inaddition, the data streaming methods of the present invention may beused to convey large databases of data which are to be displayedgraphically, such as in graphic displays of stock values. It also willbe appreciated that the embodiments described above are cited by way ofexample, and the full scope of the invention is limited only by theclaims.

We claim:
 1. A method of streaming over a network a three-dimensional representation of a three-dimensional surface, wherein three-dimensional features of said surface are represented by data which includes elevation information and image data arranged in a hierarchical structure including portions of said surface at a plurality of detail levels comprising: retrieving locally stored data that includes elevation information corresponding to a portion of said surface at a starting detail level from a local memory; determining whether said three dimensional portion of said surface is at a detail level that satisfies a calculated detail level, and interatively and dynamically streaming over a network remotely stored additional three dimensional data that includes elevation information of portions of said surface at successively higher and higher detail level until the calculated detail level is satisfied.
 2. The method of claim 1 further comprising: dynamically streaming from the server labels and annotations data associated with the portion of said surface. displaying the labels and annotations data overlaid on the three-dimensional surface.
 3. The method of claim 1 further comprising: dynamically streaming from the server vector data associated with the portion of said surface. displaying the vector data overlaid on the three-dimensional surface
 4. The method of claim 1 further comprising: dynamically streaming from the server three-dimensional polygonal data associated with the portion of said surface. displaying the three-dimensional polygonal data on the three-dimensional surface.
 5. The method of claim 2 whereby the labels and annotations that are overlaid on the three-dimensional surface are used as advertisements.
 6. The method of claim 1 further comprising: rendering a three-dimensional representation of said portions of said surface at the local computer, and repetitively performing all previous steps as the viewpoint in the scene changes position.
 7. The method of claim 1 wherein the three-dimensional representation is received at a particular computer in a network of computers.
 8. The method of claim 1 wherein the calculated detail level is based on a viewpoint position in the scene.
 9. The method of claim 1 wherein the additional data is compressed.
 10. The method of claim 1 further comprising streaming and caching excess three-dimensional or image information not currently needed.
 11. The method of claim 1 wherein said streaming includes providing one or more coordinates along with detail level information.
 12. The method of claim 1 wherein at least some of said data contains color attributes.
 13. The method of claim 1 wherein said three-dimensional data and said image data depends on a specific area and scene selected by a user.
 14. The method of claim 13 wherein the user at said local computer connects to the server, and the local computer receives sufficient data to render said specific area and scene selected by the user.
 15. A system for sending information over a network representing at least a portion of a three-dimensional surface comprising: a local computer and a remote server communicating via a network; the server having storage capability to store hierarchical sets of information representing three-dimensional features of the portion of the three-dimensional surface at multiple resolution levels; the server adapted to dynamically stream three-dimensional data that includes elevation information and image data from the hierarchical data sets representing said portion of the three-dimensional surface to the local computer based on the changes in viewpoint position within the scene; the server streaming three dimensional data representing said portion at higher and higher resolution levels requested by said local computer; the local computer adapted to render said particular scene from three-dimensional data that includes elevation information and said image data, representing said portion of the surface.
 16. The system of claim 15 wherein said local computer is adapted to iteratively request sets of information representing three-dimensional features of said portion of the three-dimensional surface from said remote computer of higher and higher resolution until resolution of said portion at said local computer satisfies a predetermined level.
 17. The system of claim 15 further comprising a cache memory and cache manager.
 18. The system of claim 15 wherein said hierarchically stored sets of information are compressed.
 19. The system of claim 15 wherein at least some of said sets of information contain color attributes. 